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REMARKS 

Overview 

The Examiner responded in the prior Office Action as follows: rejected claims 1-4, 6-15, 
17-22, 24-35, 37-46, and 49-61 under 35 U.S.C. 103(a) as being unpatentable over Bejar et al. 
(U.S. Pat. No. 6,295,439) in view of Walker et al. (U.S. Patent No. 5,862,223); rejected claims 5, 
23 and 36 under 35 U.S.C. 103(a) as being unpatentable over Bejar in view of Walker and of 
Burstein et al. (U.S. Patent No. 6,181,909); and rejected claims 47-48 under 35 U.S.C. 103(a) as 
being unpatentable over Bejar in view of Walker and of Christensen et al. (U.S. Patent No. 
5,881,230). The status of pending claim 16 is unclear, although the Examiner's comments 
appear to assert that Bejar is at least related to claim 16 - Applicants request that the Examiner 
clarify the status of claim 16 in the next Office Action, although Applicants believe that 
dependent claim 16 is patentable for at least the same reasons as the claim from which it 
depends, as discussed below. 

Applicants hereby amend claims 1, 12-14, 19, and 32 in order to clarify the subject matter 
of their invention, and further hereby add new claims 62-90. Thus, claims 1-90 are pending. 

Discussion 

The Examiner has rejected each of the previously pending claims 1-15 and 17-61 as 
being unpatentable over a combination of Walker and Bejar, either alone or in combination with 
Burstein or Christensen. 

However, various of the pending claims include features and provide functionality not 
disclosed by Walker, Bejar or the other references, and thus are allowable over those references. 
As one example, previously pending claims 13-14, 43-44, and 55 each generally recite aspects of 
tracking the accuracy of humans who perform tasks or subtasks, and using information about 
humans' past task performance accuracy in various ways when performing a current task - for 
example, previously pending claim 55 recites "wherein the required capabilities of the human for 
performance of the first subtask include a specified degree of historical accuracy by the human 
when performing subtasks". As an illustrative example, the application as filed describes a 
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situation in which, for a task using French speaking humans, each subtask is to "be performed by 
at least 10 humans with a past accuracy record of at least 90%". 

Similarly, independent claims 1, 19 and 32 as amended each generally recite using 
information about humans' past task performance in various ways when performing a current 
task. For example, independent claim 19 generally recites using past subtask performance 
accuracy of a human as part of selecting the human to perform a current subtask, including the 
following: "the first subtask for performance by one or more humans and having one or more 
associated criteria related to performance . . . including an indication of a level of past accuracy 
of a human of previously performing subtasks; retrieving information about past accuracy of a 
first human in previously performing subtasks; sending an indication of the first subtask to a 
second computing system for performance by the first human, ... the past accuracy information 
of the first human satisfying the indicated level of past accuracy for the first subtask". 
Independent claims 1 and 32 each generally recite similar language regarding using information 
about past subtask performance of a human to facilitate performance by the human of a current 
subtask. 

However, none of the cited references appear to include any teaching or motivation to 
track humans' task performance accuracy and to use past accuracy information for a human as 
part of that human's performance of other current tasks. With respect to previously pending 
claims 13-14, 43-44, and 55, the Examiner has asserted that lines 31-36 of column 13 of Bejar 
and lines 23-26 of column 19 of Bejar disclose tracking the accuracy of all evaluators and using 
the accuracy information as part of generation of a current task result. However, these portions 
of Bejar do not include any teaching or motivation to use information about the past accuracy of 
a human as part of that human's performance of a current task - instead, the cited portions 
merely indicate that information is presented to evaluators in such a way as to minimize negative 
external effects on the scoring {e.g., to minimize distractions), and that systemic differences in 
evaluators' grading scales are reported to test developers or others. These portions lack any 
indication that, for example, a human's past accuracy information is used when selecting 
whether the human is appropriate to perform a current task, and thus appear unrelated to 
previously pending claim 55 and independent claim 19 as filed as discussed above. More 
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generally, Applicants can find no other portions of Bejar or the other cited references that are 
related to using information about a particular human's past accuracy in performing tasks when 
that human is later performing other tasks. Accordingly, independent claims 1,19 and 32 are 
each patentable over the cited prior art for at least these reasons. Similarly, the pending 
dependent claims that depend from those claims are also allowable for at least the same reasons, 
as well as for additional reasons based on the claim elements of those dependent claims. 

In addition, as another example of features and functionality of some of the pending 
claims that are not disclosed by Walker, Bejar or the other relied-upon references, previously 
pending claims 12, 29 and 42 each generally recite aspects of providing a desired level of 
accuracy for results of performance of a task by having multiple humans each perform the same 
subtask and identifying a desired level of agreement between the results from the multiple 
humans. For example, previously pending claim 42 recites "dispatch information about the first 
subtask to multiple additional humans to perform said first subtask, and said accuracy is based at 
least in part on receiving results in agreement from multiple of the humans to whom the 
information about the first subtask is dispatched", and independent claim 12 as amended 
similarly recites "automatically facilitating generation of a result having desired accuracy for the 
first subtask by, determining that the received results [of performance of a first subtask] include a 
common result that was received from each of a selected number of humans, the selected number 
of humans for said accuracy being greater than 1 and based on at least one of a majority of the at 
least some humans and of at least a specified number N of two or more humans; and selecting 
the common received result as the result for the first subtask having the desired accuracy". 

However, none of the cited references appear to include any teaching or motivation to 
provide a desired level of accuracy for task performance results by having multiple humans each 
perform the same subtask and identifying a desired level of agreement between the results from 
the multiple humans. With respect to previously pending claims 12, 29 and 42, the Examiner has 
asserted that lines 24-28 of column 15 of Bejar and lines 31-37 of column 16 of Bejar disclose 
the claim elements. However, these portions of Bejar do not include any teaching or motivation 
to identify a desired level of agreement between the results for a subtask from multiple humans. 
Instead, the cited portions of Bejar merely indicate that multiple humans may evaluate a 
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particular question in order to identify systemic differences in evaluators' grading scales. 
Applicants can find no teaching or motivation in Bejar or any of the other cited references that is 
related to identifying results from multiple humans that are in agreement in order to provide 
results with a desired level of accuracy. Accordingly, independent claim 12 and dependent 
claims 29 and 42 are each patentable over the cited prior art for at least that reason, as are the 
pending dependent claims that depend from those claims. 

Furthermore, various of the pending dependent claims continue to recite additional 
features that are not taught or motivated by the relied-upon prior art references and thus are 
believed to be further allowable for those reasons as well, but are not enumerated here for the 
sake of brevity. 



In light of the above remarks, Applicants respectfully submit that all of the pending 
claims are allowable. Applicants therefore respectfully request the Examiner to reconsider this 
application and timely allow all pending claims. If the Examiner has any questions or believes a 
telephone conference would expedite prosecution of this application, the Examiner is encouraged 
to call the undersigned at (206) 694-4815. 

The Director is authorized to charge any additional fees due by way of this Amendment, 
or credit any overpayment, to our Deposit Account No. 19-1090. 
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SEED Intellectual Property Law Group PLLC 
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